Community

Prog for Finance- Data scraping S&P500 P6

Hi all/Harrison,

I'm in middle of the Prog for Fin but having 2 problems with extracting data as per the training video as follows:

1) pandas_datareader.data command is not extracting Yahoo ticker history, I had a look at the pandas_datareader docs for remote data access solution but the code on it's page is showing the same error message that I'm getting. I've circumvented this problem by using "google" finance instead which has worked fine but it doesn't give me the adjusted stock close price.

2) Using "google" api worked fine until it came to extracting the S&P 500 data into csv folders when running "get_data_from_google()", for some reason the csv data for ticker "LMT" is not available on Google finance for some reason(verified) so it's tripping out the code as the download stops at "LMT" with a "Remote Data Error".

My questions are:

1) Is there another way of getting Yahoo ticker info as the adjusted price is best price to use?
2) How can I update code in "get_data_from_yahoo()" so that it ignores any tickers that have problems or are missing data so that it avoids tripping out?

The code is:

def get_data_from_google(reload_sp500=False):
    if reload_sp500:
        tickers = save_sp500_tickers()
    else:
        with open("sp500tickers.pickle", "rb") as f:
            tickers = pickle.load(f)
            
    if not os.path.exists('stock_dfs'):
        os.makedirs('stock_dfs')
        
    start = dt.datetime(2000,1,1)
    end = dt.datetime(2017,6,30)
    
    for ticker in tickers:
        print(ticker)
        if not os.path.exists('stock_dfs/{}.csv'.format(ticker)):
            df = web.DataReader(ticker, 'google', start, end)
            df.to_csv('stock_dfs/{}.csv'.format(ticker))
        else:
            print('Already have {}'.format(ticker))
            
get_data_from_google()